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ABSTRACT 

We investigate clustering properties of Lyman-break galaxies (LBGs) at z ~ 3 based on deep 
multi-waveband imaging data from optical to near-infrared wavelengths in the Subaru/XMM- 
Newton Deep Field. The LBGs are selected by U — Uand V—z' colors in one contiguous area 
of 561 arcmin 2 down to z' — 25.5. We study the dependence of the clustering strength on rest- 
frame UV and optical magnitudes, which can be indicators of star formation rate and stellar 
mass, respectively. The correlation length is found to be a strong function of both UV and 
optical magnitudes with brighter galaxies being more clustered than faint ones in both cases. 
Furthermore, the correlation length is dependent on a combination of UV and optical magnitudes 
in the sense that galaxies bright in optical magnitude have large correlation lengths irrespective of 
UV magnitude, while galaxies faint in optical magnitude have correlation lengths decreasing with 
decreasing UV brightness. These results suggest that galaxies with large stellar masses always 
belong to massive halos in which they can have various star formation rates, while galaxies with 
small stellar masses reside in less massive halos only if they have low star formation rates. There 
appears to be an upper limit to the stellar mass and the star formation rate which is determined 
by the mass of hosting dark halos. 

Subject headings: cosmology: observations — cosmology: large-scale structure of universe — galaxies: 
evolution — galaxies: high-redshift 
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Galaxy evolution in dark halos is a central is- 
sue of cosmology in the Cold Dark Matter (CDM) 
universe. A critical key to the problem is to reveal 
what kind of dark halos host what kind of galaxies, 
specifically, the relationship between the mass of 
dark halos and fundamental quantities of galax- 
ies such as star formation rate and stellar mass. 
The clustering strength of galaxies can be used 
to infer the mass of hosting dark halos, since in 
the standard theoretical framework of galaxy evo- 
lution, the large-scale distribution of galaxies is 
determined by the distribution of underlying dark 
halos, whose spatial clustering depends on their 
mass, with more massive halos hav ing stronger 
spatial clustering (|Mo fc Whitdll996l ). 



A number of studies have examined cluster- 
ing properties of various high-rcdshift galaxies 



Giavalisco et al. | 1998: iGiayalisco fc Dickinsor 
20011 iFoucaud et alj 120031: iDaddi et all 12003 ' 



Ouchi et al.ll2004ari2005t lAdelberger et al.H2005 
Lee et al.ll2006: Kashikawa et al.l 2006: Hil debrandt et al 



20071 : iQuadri et al.1 l2007t llchikawa et alJ l2007t >~ 



Measuring clustering strength requires a large 
galaxy sample from a wide sky coverage. One 
considerable case in which the relationship be- 
tween dark halos and galaxies has been success- 
fully explored by means of clustering analysis is 
that of Lyman-break galaxies (LBGs). LBGs, 
which are selected by their continuum features 
in rest-frame UV spectra (the Lyman-break, the 
Lya forest, and otherwise nearly flat continuum 
longwar d of the Lyq) redshifted into optical band 



Stcidel et al 



passe s (jGuhathakurta et al.l Il990t 
19961 ). are normal, young star- forming galaxies 



with modest dust extinction. Requiring only op- 
tical imaging in a few bands, the LBG selection 
method has yielded the largest and well-controlled 



z < 6 (e.g.. Steidel et al. 1999. 20031: Ouchi et al. 


2004b: Giavalisco etalj £004: 


Dickinson et al. 


2004 


; ISawicki & Thompson 2006 


; Yoshida et al. 


2006 


; Iwata et al. 2007: Bouwens et al.l 2007). 



LBGs are as numerous as the present-day galax- 
ies, suggesting that they play a significant role in 
the early stage of galaxy evolution. For a review 
of LBGs including th e history of their discovery, 
see iGiavalisco ( 20021) . 

An important piece of evidence which links 
the mass of hosting dark halos to physical prop- 
erties of LBGs has been first discovered for 
the UV luminosity; the clustering strength of 
LBGs increases strongly with UV luminosity 



(e.g., Giavalisco & Dickinson 2001; Ouchi et al 
2004at lAdelberger et all l2005at iLee et al.l ^OOa 
Hildebrandt et al.ll2007h . Since UV luminosity is 



sensitive to star formation rate, this implies that 
the star formation activity of LBGs is somehow 
controlled by the mass of dark halos which host 
them. To make a next step forward in our un- 
derstanding of how the evolution of galaxies is 
related to the mass of dark halos, it would be 
necessary to examine the clustering strength as a 
function of other properties of galaxies like stel- 
lar mass, age, and dust extinction. In particular, 
stellar mass is a robust quantity in terms of the 



star formation history of galaxies compared to star 
formation rate, which can vary with time. This 
kind of analysis is, however, not easy because it 
requires deep and wide-field observational data 
covering wavelengths simultaneously from optical 
to near-infrared. Only the dependence of cluster- 
ing strength on res t-frame optical luminosity has 
been examined by lAdelberger et alJ (|2005bl ) for 
300 galaxies at z ~ 2 (BX objects) selected by UV 
wavelength in a similar manner to LBGs. 

Recently, clustering properties of near-infrared 
selected galaxies at high redshift have been ex- 
amined against va r ious quantities of galaxies 



(|Quadri et all l2007t llchikawa et al.l 120071 ). The 



selection based on a near-infrared magnitude al- 
lows us to compile a nearly stellar mass-selected 
sample. However, the studies at high redshift on 
the basis of near-infrared selected galaxies have so 
far been inevitably limited to shallow (K < 23.0; 
I Quadri et aT]|2007l) or small-field (< 25 arcmin 2 ; 
llchikawa et al.l |2007T ) samples. This is due to the 
lower sensitivity and smaller area coverage of near- 
infrared cameras compared to optical ones. 

This paper studies the dependence of clustering 
strength on both UV (apparent z' band) and op- 
tical (apparent K band) luminosities for LBGs at 
z ~ 3 to investigate how the stellar mass and the 
star formation rate of galaxies depend on the mass 
of dark halos which host them at high redshift. 
The redshift z ~ 3 is the best target for such stud- 
ies, since it is the highest redshift at which ground- 
based near-infrared imaging can access rest-frame 
optical wavelengths. To construct a large sam- 
ple of LBGs at z ~ 3, we performed £/-band 
imaging observation with Subaru/Suprime-Cam 
in the Snbava/ XMM-Newton Deep Field (SXDF), 
which has a set of very deep optical multi- 
waveband imaging data from the Subaru/XMM- 
Newton Deep Survey (SXDS; ISekiguchi et al.l 



in preparation! ) . and deep J and K data taken 
with the wide-field near-infrared camera WFCAM 
on U KIRT by the UKIDSS Ultra Deep Surve y 
(UDS; lLawrence et~ai1l2007l: I Warren et ai]|2007l ). 

The outline of this paper is as follows: In Sj^l 
a brief account of the observations and the data 
is presented. A large sample of LBGs at z ~ 3 is 
constructed in ^31 Simulations to assess the red- 
shift distribution function and the contamination 
by interlopers of the sample are also described. 
Clustering analysis is made in 21 A summary 



2 



and conclusions are given in Sj5] 

Throughout this paper, th e photometric system 
is based on AB magnitude ( Oke k. Gunnl 1983). 



The cosmology adopted is a flat universe with 
f2 m = 0.3, tt\ — 0.7, erg = 0.9, baryonic density 
fib = 0.04, and a Hubble constant of Ho — 100 h 
km s _1 Mpc -1 with h = 0.7. The correlation 
length is expressed in units of h^ 1 Mpc to facili- 
tate comparison with previous results. 

2. DATA 



We carried out J7-band imaging observations in 
the southern part of the Subaru/ XMM-Newton 
Deep Field (SXDF) [02 ,I 18 m 00 s , -05°00'00" 
(J2000)] with Subaru/Suprime-Cam in 2002. The 
survey field was covered with one single pointing of 
the Suprime-Cam, i.e., 34' x 27', with a pixel scale 
of 0."202 pixel -1 . The stacked image has a seeing 
size, PSF FWHM, of 1.35". The total exposure 
time was 3.7 hours and the la surface brightness 
fluctuation within a 1" diameter aperture is 29.07 
mag. The 5cr limiting magnitude within a 2" di- 
ameter aperture is 26.41 mag. The SXDF has a 
set of very wide and deep multi-waveband opti- 
cal imaging data from the Sub&ru/ XMM-Newton 
deep survey project fSXDS: lFurusawa et al. 2007 ) 
taken with Subaru/Suprime-Cam in five standard 
broad-band filters, B, V, R, i', and z' . We com- 
bine these data with the [/-band data to select 
LBGs at z ~ 3. All of the images were aligned 
and smoothed with Gaussian kernels so that all 
have the same PSF size as that of the [/-band 
image. The surface brightness limits (la fluctu- 
ation within a 1" diameter aperture) are 30.94, 
30.38, 30.17, 29.90, and 28.87 mag in B, V, R, i', 
and z', respectively. The 5a limiting magnitudes 
within a 2" diameter aperture are 27.88, 27.30, 
27.09, 26.77, and 25.76 mag in B, V, R, i', and 
z', respectively. 

Part of the SXDF was imaged in the J and 
K bands with UKIRT/WF CAM by the UKIDSS 



Ultra Deep Survey (UDS; lLawrence et all 12007 
Warren et al.ll2007h . In this study we make use of 



these data (DR1) to examine near-infrared prop- 
erties of LBGs at z ~ 3. The J-band and K- 
band images were also aligned and smoothed to 
be matched with the optical images. The surface 
brightness limits (la fluctuation within a 1" di- 
ameter aperture) are 26.78 and 26.56 mag in J 



and K, respectively. The 5a limiting magnitudes 
within a 2" diameter aperture are 23.67 and 23.47 
mag in J and K, respectively. 

Object detection and photomet ry were per- 
form ed using SExtractor version 2.3 ( Bertin fc Arnoutsl 
119961 ). We detected objects in the z'-band image, 
and for each detected object photometry was made 
in all the images at exactly the same position by 
running SExtractor in "double-image mode" . We 
adopt MAG.AUTO in SExtractor for total mag- 
nitudes, and use magnitudes within a 2" diameter 
aperture to derive colors0 with an aperture correc- 
tion of —0.2 mag for U magnitudes since the PSF 
shape in the U band image was different from 
those in the others. The value —0.2 was deter- 
mined so that the difference between the aperture 
magnitudes and the total magnitudes of the U 
band became equal to those of the other bands for 
LBG candidates. The magnitudes of objects were 
corrected for a small amount of foreground Galac- 



tic ext inction using the dust map of lSchlegel et al 
(|l998l) . The reddening is E(B — V) = 0.023, corre- 
sponding to extinctions of Ajj = 0.10, Ab = 0.09, 
A v = 0.07, An = 0.06, A v = 0.05, A z , = 0.03, 
Aj = 0.02, and A K = 0.01. 

Clustering analysis requires high uniformity in 
sensitivity over the whole area, since fluctuations 
of sensitivity can produce spurious clustering sig- 
nals and bias measurements of clustering strength. 
We examined the sensitivity variation over the 
area of the images of respective bands by divid- 
ing them into small meshes and estimating the 
sky noise in each of the meshes. Based on these 
sky-noise maps, we carefully defined a high-quality 
region in which the sensitivity is good and uni- 
form, trimming the edges of the images where sky 
noise was systematically larger due to dithering 
observation. The effective area with a complete 
coverage in all of the six optical bands amounts 



1 We also measure magnitudes within a larger aperture of 
3" diameter, as the PSF FWHM of the final images is 
somewhat large. However, since over 90 % of the LBG 
candidates selected with 2"-aperture magnitudes overlap 
with those selected with 3"-aperture magnitudes, we adopt 
2"-aperture magnitudes in order to obtain colors of faint 
objects with better S/N. In addition, we compare aper- 
ture magnitudes with isophotal magnitudes and find that 
the isophotal areas of objects satisfying the magnitude 
threshold for LBG selection (23.0 < z' < 25.5) are mostly 
larger than the 2"-diameter aperture. This means that 
2"-aperture magnitudes are less noisy than isophotal mag- 
nitudes for most objects. 
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Fig. 1. — The U, V, and z' bandpasses overplot- 
ted on the spectrum of a generic z — 3 galaxy 
(thick line), illustrating the utility of color selec- 
tion technique using these three bandpasses for lo- 
cating z ~ 3 galaxies. 

to 740 arcmin 2 , and the overlapping area of the 
optical images and the near-infrared images is 
561 arcmin 2 after the low-quality regions are dis- 
carded. 

Spectroscopic follow-up observations have been 
carried out for many objects in t he SXDF with 
Subaru/FOCAS andVLT/VIMOS (lAkivama et all 



in preparation! : Simpson et alJlrn preparation! : Saito et al 



2007). The number of spectroscopically observed 
objects which are located in the region used in 
this study and have magnitudes in the range of 
23.0 < z' < 25.5, within which we select LBGs, is 
63. 



3. LYMAN-BREAK GALAXY SAMPLE 

AT z ~ 3 

3.1. Selection of Lyman-break Galaxies 

We find that a combination of U, V, and z' 
bands works best to select LBGs at z ~ 3 among 
our bandpasses set (Figure [J). Note that the U- 
band filter of the Suprime-Cam is significantly red- 
der than standard [/-band filters. Consequently, 
the mean redshift of LBGs selected is higher than 
those of traditional i7-drop LBG samples (see 
i j3.2p . In Figure [H we show the distribution of 
detected objects with 23.0 < z' < 25.5 in the 
U— Vvs V — z' diagram, as well as predicted posi- 
tions of high-redshift galaxies and foreground ob- 
jects (lower- redshift galaxies and Galactic stars). 
When the magnitude of an object in U and/or V is 
fainter than the la magnitude, the la magnitude 
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Fig. 2. — U — V vs V—z' diagram for the de- 
tected objects with 23.0 < z' < 25.5. When 
the U and/or V magnitude of an object is fainter 
than the la magnitude of the band, the la mag- 
nitude is assigned to the object. The predicted 
colors of model galaxies and stars are overplotted. 
The blue and red solid lines indicate the tracks 
for model spectra of young star-forming galaxies 
of age 0.1 Gyr and 1.5 Gyr, respectively, with red- 
dening of E(B - V) = 0, 0.15, and 0.3 (from left 
to right). The redshift range is from z — 2 to 
higher redshifts, and the circles on the track mark 
the redshift interval of 0.1 with the enlarged cir- 
cles corresponding to z = 2.5, 3.0, and 3.5. The 
green dotted, dashed, and dot-dashed lines delin- 
eate the tracks for model spectra of local ellip- 
tical, spiral, and irregular galaxies, respectively, 
redshifted from z = to 2 without evolution. The 
circles on each track mark z — 0, 1, and 2. The 
yellow asterisks re present the colors of 175 Galac- 
tic stars given by iGunn fc Strvkerl ( 1983h . The 
thick orange line indicates the boundary which we 
adopt for the selection of z ~ 3 LBGs. 
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Fig. 3. — UV luminosity function for LBGs at z ~ 
3. Our data are shown by the filled circles. The 



open squar es and crosses are from ISteidel et al 
(|l999l ) and ISawicki fc Thompson! (|2006|), respec- 
tively. 



is assigned. 

We set the selection criteria for LBGs at z ~ 3 

as: 

23.0 < z' < 25.5, (la) 
U-V>0.8, V-z'<2.7, 

U - V > 1.8(V - z') + 1.6. (lb) 

The boundaries on the U—V vs U — z' diagram 
defined by these color criteria are outlined with 
the thick orange line in Figure O The number of 
LBG candidates selected is 962 in total, among 
which 708 are found in the region observed in the 
J and K bands. 

Among the 63 spectroscopic objects, 4 objects 
satisfy the selection criteria and all of them are 
identified to be at z > 2.9. On the other hand, 
there are two additional objects which are found in 
the redshift range of 2.9 < z < 3.7 but do not pass 
the criteria. These two are fainter in U than the 
la magnitude and are not red enough in XJ\ a — V 
color. The possibility of missing targeted galaxies 
for this reason is taken into account in a simula- 
tion for estimating the completeness of our sam- 
ple described in c|3.2l The missed objects have z' 
magnitudes of z' — 24.93 and 25.13. In fact, the 
simulation shows that a small fraction of z ~ 40% 
of galaxies at 2.9 < z < 3.7 with these magnitudes 



Fig. 4. — Sky distribution of the 962 LBGs in our 
sample. The region outlined by the thick line in- 
dicates the area observed in the J and K bands, 
while the rest is the area observed only in op- 
tical bands, after removal of low-quality regions. 
The large and small circles represent LBGs with 
23.0 < z' < 24.5 and 24.5 < z' < 25.5, respec- 
tively. LBGs having K magnitudes are further 
distinguished by z' and K magnitudes with differ- 
ent colors, where red, green, cyan, and blue repre- 
sent LBGs with 23.0 < z' < 24.5 and K < 23.46, 
24.5 < z' < 25.5 and K < 23.46, 23.0 < z' < 24.5 
and K > 23.46, 24.5 < z' < 25.5 and K > 23.46, 
respectively. The projected comoving scale of 10 
Mpc at z — 3.3 is shown at the top left. North is 
up and east is to the left in this image. 



is selected by the criteria. 

The luminosity function for the LBG sam- 
ple at z ~ 3 is d e rived in the same manner 
as m lYoshida et all (|2006l ) using the complete- 
ness and the contamination fraction obtained in 
the following subsections (|0L2] and ^3.3[) . Fig- 
ure [3] shows the luminosity functio n in compar- 
ison with those in the literature ( Steidel et al 



19991 : ISawicki fc Thompson! l2006f ). Note that the 
absolute UV luminosities of our galaxies are based 
on apparent z' magnitudes, while others are based 
on apparent R magnitudes. The agreement of our 
luminosity function with those in previous studies 
is very good, particularly when one considers dif- 
ferent sample selections. Figure 2] shows the sky 
distribution of the LBG sample. 



5 




3 3.5 
Redshift 



■1.5 



Fig. 5. — Redshift distribution functions, N(z), 
of the LBGs with different apparent magnitudes 
estimated from Monte Carlo simulations. The 
red, magenta, green, cyan, and blue lines denote 
the N(z) for z' = 23.25, 23.75, 24.25, 24.75, 
25.25, respectively. The thick black line indi- 
cates the magnitude- weighted redshift distribution 
function. 

3.2. Redshift Distribution Function 

The redshift distribution function of the LBG 
sample is estimated as a function of magnitude 
through a Monte Carlo simulation. In the simula- 
tion, we generate artificial LBGs over an apparent 
magnitude range of 23.0 < m z > < 25.5 with an 
interval of Am = 0.5, and over a redshift range of 
2.0 < z < 4.5 with an interval of Az = 0.1. Model 
spectra of the artificial LBGs are constructed us- 
ing the stellar population synthesis code devel- 
oped by iKodama fc Arimotd (|1997h . As model 
parameters, an age of 0.1 Gyr, a Salpeter initial 
mass function, and a star-formation timescale of 
5 Gyr are adopted, and five values of reddening, 
E(B -V) = 0.0, 0.1, 0.2, 0.3, and 0.4, are applied 
using the dust extinction form ula for starburst 
galaxies bv lCalzetti et al.l (2000). These values re- 
produce the average rest-frame ultraviolet-optical 
spectral energy distribution of LBGs observed at 



3 (jPapovich et al.l 120011 ) . The absorption 



due to the intergalactic medium is applied follow- 
ing the prescription by iMadaiJ (119951 ). The col- 
ors are calculated by convolving thus constructed 
model spectra with the response functions of the 
Suprime-Cam filters. We assume that the sur- 



face brightness distribution of LBGs is Gaussian 
and assign apparent sizes to the artificial LBGs so 
that their size distribution measured by SExtrac- 
tor matches that of the observed LBG candidates. 
The artificial LBGs are then distributed randomly 
on the original images after adding Poisson noise 
according to their magnitudes, and object detec- 
tion and photometry are performed in the same 
manner as done for real objects. A sequence of 
these processes is repeated 100 times to obtain 
statistically accurate values of completeness. In 
the simulation, the completeness for a given ap- 
parent magnitude, redshift, and E(B — V) value 
can be defined as the ratio in number of the sim- 
ulated LBGs which are detected and also satisfy 
the selection criteria, to all the simulated objects 
with the given magnitude, redshift, and E(B — V) 
value. We calculate the completeness of the LBG 
sample by taking a weighted average of the com- 
pleteness for each of the five E(B — V) values. The 
weight is taken using the E(B — V) distribution 
function of z ~ 4 LBGs derived by lOuchi et al 



( 2004bh (open histogram in the bottom panel of 
their Fig. 20), which has been corrected for incom- 
pleteness due to selection biases. The resulting 
completeness, p(m,z), is shown in Figure O 

The magnitude-weighted redshift distribution 
function of our LBG sample is derived from 
p(m, z) by averaging the magnitude-dependent 
completeness weighted by the number of LBGs in 
each magnitude bin (a thick black line in Figure^]). 
The average redshift, z, and its standard devia- 
tion, s z , are calculated to be z = 3.3 and s z = 0.3. 
As we mentioned in §3.1[ due to the redder [/-band 
filter of the Suprime-Cam, the redshift distribu- 
tion function is biased toward higher redshifts in 
comparison with traditional [/-drop LBGs, whose 
redshifts are distributed around z ~ 3.0 (e.g., 



ISteidel et al.ll2003l ). 

The measurement of the correlation length from 
the angular correlation function relies on the esti- 
mation of the redshift distribution function (Q. 
In order to explore to what extent the redshift 
distribution function and the resultant correlation 
length are affected by the model assumption, we 
recalculate the redshift distribution function us- 
ing a different model spectrum with an age of 1.5 
Gyr. In fact, some of the LBGs in our sample 
have red z' — K colors explained by such spectra. 
The derived correlation lengths based on the two 
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Fig. 6. — Fraction of interlopers as a function of 
magnitude for the LBG sample. 



different model spectra are found to be consistent 
within the errors. 

3.3. Contamination by Interlopers 

We estimate the fraction of low-redshift inter- 
lopers in the LBG sample also by a Monte Carlo 
simulation as follows. The adopted boundary red- 
shift between interlopers and LBGs is zq = 2.9. 
We use objects in the Hubble Deep Field North 
(HDFN), for which best-fi t spectra and ph o tomet - 
ric redshifts are given by iFurusawa et al~ ( 2000h . 
as a template of the color, magnitude, and redshift 
distribution of foreground galaxies, and generate 
929 artificial objects which mimic the HDFN ob- 
jects. The apparent sizes of the artificial objects 
are adjusted so that the size distribution recov- 
ered from the simulation is similar to that of the 
real objects in our catalogs. We distribute the 
artificial objects randomly on the original images 
after adding Poisson noise according to their mag- 
nitudes, and perform object detection and pho- 
tometry in the same manner as employed for real 
objects. A sequence of these processes is repeated 
100 times. In the simulation, the number of inter- 
lopers can be defined as the number of the sim- 
ulated objects with low redshifts [z < zo) which 
are detected and also satisfy the selection criteria 
for LBGs. The number of interlopers expected in 
the LBG sample can then be calculated by multi- 
plying the raw number by a scaling factor which 
corresponds to the ratio of the area of our field 
(740 arcmin 2 ) to the area of the HDFN multiplied 
by the repeated times (100 x 3.92 arcmin 2 ). Fig- 
ure[6]shows the fraction of interlopers for our LBG 



sample as a function of magnitude. The fraction 
is found to be at most 6% at any magnitude. 

4. CLUSTERING PROPERTIES 
4.1. Method 

We measure the angular correlation function 
(ACF), u>(9), u s ing t he estimator proposed by 



Landv fc Szalavl (|l993h : 



DD(9)-2DR(9)+RR(9) 
= ^ , (2) 

where DD(9), DR(9), and RR(9) denote the 
numbers of galaxy-galaxy, galaxy-random, and 
random-random pairs, respectively, with angular 
separations between 9 — 59/2 and 9 + 59/2. The 
distribution of random points is subject to ex- 
actly the same geometry as the observed area, 
avoiding regions where galaxies are not detected, 
e.g., in the vicinity of bright stars. We generate 
100 times as many random points as the number 
of galaxies in order to reduce the uncertainties. 
The erro rs are estimated by bootstrap resampling 
method ( Ling et al]|l986f h As the contamination 
is very small for our LBG sample, we do not apply 
any correction for it. 

The ACF can be approximated as a power law 
given by 



u(0) 



A,9- 



(3) 



However, since the average number of galaxies in 
a given field is estimated from a sample itself and 
fluctuations on the scale of the field size are not ac- 
counted for, the measured ACF is underestimated 
by a constant known as the integral constraint, IC 
(|Groth fc Peebledll977l) : 



^obs(^) 



tu truc (9) - IC 



A t „9 



-0 



IC.(4) 



The value of the integral constraint is equal to the 
variance of the number of galaxies in the field: 



IC = 



1 



N. 



sal 



(5) 



where the first term is the Poisson variance and 
the second term accounts for an additional vari- 
ance caused by clustering. The variance caused by 
clustering can be estimated by integrating uj tme (9) 
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over the survey area ( Roche fc Earledll999h : 



Wtrue(^) d£L\d£l2 



1 



(6) 



The quantity a^/A^ i s estimated directly from 
the random point catalogs for a given value (3. 
Then the amplitude of the ACF, A u> can be es- 
timated through x 2 fitting of w b s (#) using equa- 
tions (U]) and ([5]). Followin g recent results in clus- 
tering studies o f LBGs (jAdelberger et al.l 12005a : 
Lee et al. 20061) . we assume /3 = 0.6 in what fol- 
lows. The fitting is made only at 9 > 10" since 
at 9 < 10" there may be a significant contribution 



2005; Lee et al 



from nonlinear small s cale clustering (jOuchi et al 
^ 20061) 



The error in A u 



is esti- 
mated from the range in which an increase in \ 2 
from the best-fit value is less than unity. 

We derive the spatial correlation function, £(r), 
by inverting lo(9) using the Limber transform 
(|Peebleslll980i) . If the ACF is a power law, the spa- 
tial correlation function also has to have a power 
law form: 



(7) 



= - 

So 



where rg is the spatial correlation length and 7 
(3 + 1. The is related to r as 



A 



Crl I F{z)D 1 -\z)N{zfg{z)dz 

n -2 



N{z)dz 



(8) 



where Dg is the angular diameter distance, N(z) 
is the redshift distribution function, 

g{z) = ^{(i + z) 2 (i + n z + n A [(i + z)- 2 - 



F(z) = [(l + z)/(l + z)] (3+£) . We assume con- 
stant clustering in comoving units in the redshift 
range of our LBG sample; in this case the param- 
eter e is specified by e = 7 — 3. For the redshift 
distribution function, we use the one obtained by 
the simulation in H3.2I 

The standard CDM model predicts that the 
cluste ring of dark halos is correlated with halo 
mass ( Mo fe White 1996f h We use the observed 
spatial correlation function of LBGs to infer the 
mass of dark halos hosting t hem on the b asis o f 
the analytic model given by ISheth et al. ( 2001 ) , 
which is derived from a fit to large N-body simu- 
lations. According to the model, the bias of dark 
halos, &dh, which relates the clustering of dark ha- 
los to that of the overall dark matter, is calculated 

by 



5DH 



V 

: 0.5, C 



1 + 



1 r 



v 12 + bv' 2 ^ 

,,'2c, 



v' 2c + b{l- c)(l- c/2) 



,(9) 



: \fav, and the constants a 
0.6. Here, v is defined by 



v — 



a(M, z) D(z)a(M, 0) 



0.707, 



(10) 



and 



where D(z) is the growth factor, a(M,z) is the 
relative mass fluctuation in spheres that contain 
an average mass M, and S c ss 1.69 represents the 
critical amplitude of the pert urbation for collapse . 
We calculate D(z) following ICarroll et alj (|l992f ) 
and er(M, 0) from the initial power spectrum with 
a pow er law index of n = 1 u sing the transfer func- 
tion of lBardeen et al. (1986). Since v' is a function 
of redshift and mass, the mass of dark halos is es- 
timated from equation |9]), once the bias of the 
dark halos and the redshift are given. We assume 
that the observed bias of galaxies at a large scale 
reflects the bias of dark halos hosting them (i.e., 
6 ga i ~ &dh)i thereby obtain an estimate of the 
..\-| 1/2 Costing halo mass. The bias of galaxies against 
ilark matter at a large scale (— 8 /i _1 Mpc) is mea- 
sured by 



C 



- r[(7-i)/2] 
r(7/2) 



The function F(z) describes the evolution of £(r) 
with redshift. The evolution is often modeled as 



/ g(8 Zi-iMpc) 
£dm(8 /i-'Mpc) 

; [(8 h-mpc)/^' 
Cdm(8 h-mpc) 



(11) 
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Fig. 7. — Angular correlation function, u>(0), for 
the z ~ 3 LBG sample. 



where £dm is the predicted spatial correlation 
function of the overall dark m atter and is com- 
puted by the nonlinear model of Peacock fe Doddsl 
f)l996h . 



4.2. Results 

Figure[7]shows the ACF for the full LBG sample 
containing 962 galaxies with 23.0 < z' < 25.5. We 
calculate the spatial correlation length from the 
amplitude of the ACF using equation (jHJ) to find 
r = 5.5±^ h^Mpc (comoving; Table |IJ. The 
average mass of dark halos hosting these LBGs is 
estimated to be Mdm ~ 2 x 10 12 M Q from equa- 
tions ((9]), (fl"0|) . (fTT|) . We compare the measure- 
ment of the spatial correlation length with those 
from other authors for sample s with similar red 
shift and luminosity ranges. iHildebrandt et al 
(<2007l) found r = 5.0±g'J h^Mpc for LBGs with 
23.5 < R < 25.5, which is consistent with ours. 



On the other ha nd, lAdelberger et all (|2005al ) and 
Lee et al. (2006) found slightly smaller values of 
r = 4.0±g;| /i _1 Mpc for LBGs with 23.5 < R < 
25.5 and r = 4.1+°;* fr^Mpc for LBGs with 
R < 25.5, respectively. This may be due in part to 
the fact that the redshift distribution function of 
our LBG sample is biased toward higher redshifts 
as discussed in §3.21 The average redshift of Adel- 
berger et al.'s (2005a) sample is z = 2.9, while 
ours is z — 3.3. At a given magnitude limit, this 
means that our sample includes brighter LBGs, 
which are more strongly clustered. In addition, at 
a fixed luminosity, LBGs at higher redshifts are 
more strongly clustered. We also note that our 
sample is z'-magnitude limited, while the others 



Fig. 8. — Angular correlation function, u>(0), for 
three subsamples selected by z' magnitude. The 
red, green, and blue symbols indicate the uj{0) of 
LBGs with 23.0 < z' < 24.5, 24.0 < z' < 25.0, 
and 24.5 < z' < 25.5, respectively. 



are i?-magnitude limited. The distribution in the 
z' vs R diagram of our sample implies that there 
might be more galaxies with z' > 25.5 than those 
with R > 25.5, suggesting that our sample may 
systematically miss more faint galaxies than the 
other samples. This could result in our larger r$. 

In the following, we measure the spatial corre- 
lation length for various subsamples to investigate 
the relationship between clustering strength and 
galaxy properties. 

4-2.1. Dependence of Clustering Strength on UV 
Luminosity 

It is reported that the clustering strength of 
LBGs depends on the rest-frame UV luminos- 
ity in the sense that galaxies with higher UV lu- 
minos ities have larger spatial corre l ation lengths 
e.g., iGiavalisco fc Dickinson! 2001 ; Ouchi et al 



2004at lAdelberger et alj |2005al iLee et al.l l200a 
Hildebrandt et all 120070 . We examine the clus- 
tering segregation with respect to the rest-frame 
UV luminosity in our sample using subsamples 
selected by z' magnitude (23.0 < z' < 24.5, 
24.0 < z 1 < 25.0, and 24.5 < z 1 < 25.5) from 
the full sample (Table [J). The ACF for each sub- 
sample is shown in Figure [8] In Figure [9l the 
derived correlation length is plotted as a function 
of z' magnitude. The magnitude of each point in 
Figure [5] is the median magnitude of the corre- 
sponding subsample. In agreement with previous 
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Fig. 9. — Spatial correlation length, 
tion of z' magnitude. 
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Fig. 10. — Angular correlation function, uj{9), for 
three subsamples selected by K magnitude. The 
red, green, and blue symbols indicate the oj(9) of 
LBGs with K < 23.46, 22.96 < K < 23.96 and 
K > 23.46, respectively. 



studies, we find that the correlation length in- 
creases with rest-frame UV luminosity, suggesting 
that galaxies with higher star formation rates are 
hosted by more massive dark halos. 

4-2.2. Dependence of Clustering Strength on Op- 
tical Luminosity 

We divide the sample by K magnitude (K < 
23.46, 22.96 < K < 23.96, and 23.46 < K) to see 
clustering dependence on the rest-frame optical lu- 
minosity (~ 5000A), which can be an indicator of 
stellar mass (Table [TJ). The ACF for each sub- 
sample is shown in Figure 1101 Figure [TT] shows 
the relationship between the derived correlation 
length and K magnitude. The magnitude of each 
point in Figure [TT] is the median magnitude of the 
corresponding subsample. It is found that there 
is a strong trend that LBGs brighter in rest-frame 
optical are also more strongly clustered. 

Our LBG sample shows a correlation between 
K magnitude and z' — K color; there is a sig- 
nificant deficit of bright LBGs with blue z' — K 
colors (Figure [T2"]) . Consequently, we find that 
correlation length increases with z 1 — K color 
from r = 5.4+^ /i^Mpc for z' - K < 0.8 to 
r = /i- x Mpc for z' - K > 0.8 (Table [TJ. 

Both results on the dependence of the cluster- 
ing strength imply that galaxies wi th large stellar 
masse s reside in massive dark halos. IShaplev et al 



(|2005l ) suggested a relationship between stellar 
mass and R — K color as well as K magnitude 



for their sample of BX objects at z ~ 2, and 
Adelberger et al. ( 2005VJ ) ascribed a similar seg- 
regation of the clustering strength with K magni- 
tude and R — K color for the sample to an under- 
lying correlation with stellar mass. 

4-2.3. Dependence of Clustering Strength on a 
Combination of UV and Optical Luminos- 
ity 

To investigate in what manner the star forma- 
tion activity of LBGs relates to the mass of dark 
halos in detail, we explore the dependence of the 
clustering strength on a combination of z' and K 
magnitudes, i.e., the behavior of the clustering 
strength on the z' vs K plane. The LBG sam- 
ple is divided into four subsamples on that plane 
(23.0 < z' < 24.5 and K < 23.46, 24.5 < z' < 24.5 
and K < 23.46, 23.0 < z' < 24.5 and K > 23.46, 
and 24.5 < z' < 24.5 and K > 23.46; Figure El 
Table [JJ. The magnitude of K = 23.46 corre- 
sponds to S/N — 5 in the K band image. The 
ACFs for all the subsamples are presented in Fig- 
ure [TTJ In this figure, we see that only one sub- 
sample which is faint in both z' and K has a 
smaller correlation amplitude, whereas the other 
subsamples which are bright cither in z' or K have 
very similar large correlation amplitudes. The cor- 
relation lengths derived from the amplitudes are 
plotted as functions of z' and K magnitudes in 
Figure 1151 The magnitude of each point in Fig- 
ure [TS] is the median magnitude of the correspond- 
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Fig. 11. — Spatial correlation length, ro, as a func- 
tion of K magnitude. 

ing subsample. The correlation length of galaxies 
bright in cither magnitude is large irrespective of 
the other magnitude. On the other hand, galax- 
ies faint in one of the two magnitudes have cor- 
relation lengths decreasing with the other magni- 
tude. The same trends are seen when we further 
divide the faintest subsample into two by each 
of the magnitudes, although the errors are large 
due to poor statistics. The mass of dark halos for 
these subsamples ranges from 2 x 1O 13 M for the 
subsamples with large correlation amplitudes to 
1 x 10 12 Mq for the subsample with small one. 

The overall clustering behavior against mag- 
nitudes and a color on the z' versus K plane 
suggests that the clustering strength is primar- 
ily related to the combination of K magnitude 
and z' magnitude instead of the combination of 
K magnitude and z' — K color or z' - if color 
alone. For reference to results obtained from near- 



infrared selected galaxies. iQuadri et al.1 (|2007l) ex- 
amined clustering properties of if -selected galax- 
ies (K < 22.8) at 2 < z < 3.5 and claimed 
that the clustering strength seemed to be inde- 
pendent of K magnitude for their sample. This 
does not, however, necessarily conflict with our 
result, since the limit of K magnitude of their 
sample is brighter than the limit of our bright- 
est subsample. This is true even for the absolute 
magnitudes where the slight difference of redshift 
range is taken into account. When we explore 
the dependence among our brightest subsample 
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Fig. 12.— z'-K plotted against K for the LBG 
sample. When the K magnitude of an LBG is 
fainter than the la magnitude, the la magnitude 
is assigned (open circles). Just for a guide, the 
faint limit to the sample of Ztotal < 25.5 is indi- 



cated with the dotted line. Note that the 



-K 



colors are measured with 2" -aperture magnitudes 
and some objects lie beyound the line. 



by splitting it into two, the difference in cluster- 
ing strength between the two subsamples is found 
not to be significant t hough there are large errors 

mea- 



due to poor statistics. Ilchikawa et al 



sured the clustering strength of if-selected galax- 
ies down to K — 25.0 for a deeper but 28 arcmin 2 
field and found that the clustering strength does 
increase with if-band luminosity at K > 23.0. It 
is indicated that the stellar mass of galaxies may 
not be a strong function of the mass of dark halos 
in the most massive dark halos. As to the depen- 
dence on rest-frame UV luminosity, IQuadri et al 



(|2007l ) found that the optically brighter subsample 
clusters less strongly than the fainter subsample 
for the if -selected galaxies. However, the subsam- 
ple that had a larger correlation length is fainter 
than R — 25.5, which is close to the limit of our 
sample and we cannot confirm their results from 
our sample. They have not explored the depen- 
de nce for R < 25.5. W e note here that the samples 
of IQuadri etall (|2007l ) and Ilchikawa et alJ (|2007t ) 
are rest-frame optical selected, while ours is rest- 
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Fig. 13. — Distribution of the LBG sample on the 
z' vs K plane. When the K magnitude of an LBG 
is fainter than the la magnitude, the ler magni- 
tude is assigned (open circles). The solid lines in- 
dicate the magnitude limit to the sample. The 
dashed lines indicate the boundary of four sub- 
samples. 



frame UV selected, and the differences in results 
may be partly caused by the difference in the way 
of selection. 

One implication of the result in terms of the 
stellar mass assembly of galaxies is that mas- 
sive halos can host any galaxies from ones with 
small stellar masses to ones which have accumu- 
lated large stellar masses, while in less massive 
halos, only galaxies with small stellar masses re- 
side. There appears to be an upper limit to the 
stellar mass accumulated which is determined by 
the mass of hosting dark halos. Moreover, galax- 
ies in massive halos can have various star forma- 
tion rates whatever stellar masses they have at 
z ~ 3. On the other hand, galaxies in less mas- 
sive halos accumulated only small stellar masses 
and have lower star formation rates at z ~ 3. 



Ichikawa et al.l (|2007l ) suggest a similar tendency 
for galaxies with small stellar masses in their K- 
selected galaxy sample that galaxies red in rest- 
frame UV color (galaxies with passive star forma- 
tion) in low-mass samples tend to belong to less 
massive halos, although their result was derived 
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Fig. 14. — Angular correlation function, oj{9), for 
four subsamples selected by z 1 and K magnitudes. 
Colors of the symboles for subsamples are desig- 
nated as follows: (red) 23.0 < z' < 24.5 and K < 
23.46, (green) 24.5 < z' < 25.5 and K < 23.46, 
(cyan) 23.0 < z' < 24.5 and K > 23.46, and (blue) 
24.5 < z 1 < 25.5 and K > 23.46. 



based on a small field and contains large errors. It 
is suggested that the mass of dark halos governs 
the current star formation activity as well as the 
past star formation history at z ~ 3. 

5. SUMMARY AND CONCLUSIONS 

We have investigated clustering properties of 
Lyman-break galaxies (LBGs) at z ~ 3 based on 
deep multi-waveband imaging data from optical 
to near-infrared wavelengths in the Subaru/XMM- 
Newton Deep Field. The LBGs are selected by 
U — Vand V — z' colors in one contiguous area of 
561 arcmin 2 down to z' = 25.5. The number of 
LBG candidates detected is 962 in total, among 
which 708 are found in the region observed in the 
J and K bands. We use Monte Carlo simulations 
to estimate the redshift distribution function and 
the fraction of contamination by interlopers of the 
LBG samples. The fraction is found to be at most 
6% at any magnitude. 

We explored the dependence of the clustering 
strength on rest-frame UV and optical magni- 
tudes, which can be indicators of star formation 
rate and stellar mass, respectively. The correlation 
length is found to be a strong function of both UV 
and optical magnitudes with brighter galaxies be- 
ing more clustered than faint ones in both cases. It 
is found that the correlation length also increases 
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Moreover, galaxies in massive halos can have var- 
ious star formation rates whatever stellar masses 
they have at z ~ 3. On the other hand, galaxies 
in less massive halos accumulated only small stel- 
lar masses and have lower star formation rates at 
z ~ 3. It is suggested that the mass of dark halos 
governs the current star formation acitivity as well 
as the past star formation history at z ~ 3. 
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Fig. 15. — Spatial correlation length, ro, in de- 
pendence on z' and K magnitudes. Colors of the 
symbols are the same as in Figure [TJJ 



with z' — K color, which is correlated with K mag- 
nitude in our LBG sample. These results imply 
that galaxies with larger star formation rates and 
larger stellar masses are hosted by more massive 
dark halos. 

Furthermore, the correlation length is interest- 
ingly dependent on a combination of UV and opti- 
cal magnitudes in the sense that galaxies bright in 
optical magnitude have large correlation lengths 
irrespective of UV magnitude, while galaxies faint 
in optical magnitude have correlation lengths de- 
creasing with UV magnitude. One implication of 
this result in terms of the stellar mass assembly of 
galaxies is that galaxies which have accumulated 
large stellar masses always belong to massive ha- 
los in which they can have various star formation 
rates, while galaxies with small stellar masses re- 
side in less massive halos only if they have low star 
formation rates. To put it another way, massive 
halos can host any galaxies from ones with small 
stellar masses to ones with large stellar masses, 
while in less massive halos, only galaxies with 
small stellar masses reside. There appears to be an 
upper limit to the stellar mass accumulated which 
is determined by the mass of hosting dark halos. 
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Table 1 
Clustering Measurements 



Sample 


N* 




r (ft _1 Mpc) 


23.0 <z' < 25.5 


962 


1 17 +0.29 
lml 1 -0.29 


c c+0.8 
°-°-0.9 


23.0 < z' < 24.5 
24.0 < z' < 25.0 
24.5 < z' < 25.5 


288 
467 
674 


44 +i.oo 

1 46+ 056 
1.4D_ 56 

0.761°;*° 


10.7l 2 ;? 
6.31^ 


K < 23.46 
22.96 < K < 23.96 
23.46 < K 


171 

225 
537 


3.85±i;i| 
1.80±S;g 
0.9618:1 


ii-5±3:S 


z'-K< 0.8 
z'-K> 0.8 


365 
343 


1 14+0-52 

2 cn+0.78 


5.41:- 
8.8±} : g 


23.0 < z' < 24.5, K < 23.46 
24.5 < z' < 25.5, K < 23.46 
23.0 < z' < 24.5, 23.46 < K 
24.5 < z' < 25.5, 23.46 < K 


105 
66 
96 

441 


4.63 + |^ 

o qfi+3.29 

4 41+2-49 
^•^ 1 -2.49 

qi+ - 51 


12.9+2 f 
12.51*;° 



* Number of LBGs contained in the sample. 
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